Method of determining motion vectors for a bi-predictive image block

ABSTRACT

In the method of determining motion vectors for a bi-predictive image block, reference index information from a co-located image block for the bi-predictive image block is obtained. The motion vectors of the bi-predictive image block are determined based on a type of reference picture referred to by the reference index information.

DOMESTIC PRIORITY INFORMATION

This is a divisional of U.S. application Ser. No. 10/337,611 filed Jan.6, 2003; the contents of which are hereby incorporated by reference intheir entirety.

FOREIGN PRIORITY INFORMATION

The present invention claims priority under 35 U.S.C. 119 on KoreanApplication No.10-2002-0024470 filed May 3, 2002 and Korean ApplicationNo.10-2002-0076899 filed Dec. 2, 2002; the contents of which are herebyincorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a moving picture coding system, andmore particularly, to a moving picture coding method to enhance codingefficiency.

2. Description of the Related Art

It is desirable to detect a scene change in a sequence of movingpictures so as to compress and code a moving picture sequence optimally.This is because many video applications, such as the news, sportsbroadcasting, a close-up conversation like an interview, multi-pointvideo conferencing, etc., include repeated scene changes. A scene changecan occur for an entire picture or in some area of the picture.

The digital image coding method may be changed whenever a scene changeis detected. For example, since similarity is very low between a picturein a new scene and a picture in a previous scene, a picture after ascene change is coded using an intra mode in which a picture is codedusing prediction only from decoded samples within the same picturerather than by an inter mode in which a picture is coded by motioncompensation from previously-decoded reference pictures.

In more detail, a picture in which a scene change occurs in the entirepicture, is an intra picture that is coded in the intra mode on allimage blocks forming the picture. Meanwhile, in the case of a picture inwhich a scene change occurs at some area, all blocks within the areas inwhich scene changes occur are coded in the intra mode. Since the intramode coding generates more bits as compared with inter mode coding, asequence in which scene changes occur very frequently results in low bitrate coding.

Inter mode coding may produce P pictures and B pictures. Generally, whenusing a B picture in a moving picture coding system, the coding order isdifferent from the displaying order.

FIG. 1 illustrates an example display order including B pictures. Asshown in FIG. 1, an intra picture I is displayed first among pictures tobe displayed. Two B pictures B1 and B2 are displayed after the intrapicture I. A P picture P3 is displayed after the B pictures B1 and B2are displayed. The fourth and fifth B pictures B4 and B5 are displayedafter the P picture P3 is displayed, and subsequently, a P picture P6 isdisplayed. This pattern continues to repeat until the next I picture.

However, the coding order of a digital image is not the same as thedisplay order. In other words, the P picture is coded prior to the Bpicture.

FIG. 2 illustrates an example coding order associated with the displayorder of FIG. 1. As shown in FIG. 2, if an intra picture I is coded, theP picture P3 is coded and then the two B pictures B1 and B2 that aredisplayed prior to the P picture P3 are coded. After that, pictures P6,B4, B5, P9, B7, B8, P12, B10 and B11 are coded.

Here, the B pictures have five coding modes such as intra mode, forwardmode, backward mode, bi-predictive mode and direct mode. Thebi-predictive mode uses two reference pictures. The two referencepictures are both located prior to or after the B picture or one of themis located prior to the B picture and the other is located after the Bpicture.

The direct mode utilizes temporal redundancy to maintain motioncontinuity between two adjacent pictures. In other words, in the directmode, a forward motion vector and a backward motion vector of the directmode for the B picture are derived from the motion vector of aco-located block in a subsequent picture located just after the Bpicture. Such a direct mode does not need overhead bits such as motioninformation so that the bit rate can be reduced.

Here, the forward motion vector MV_(f) and the backward motion vectorMV_(b) of the conventional direct mode are obtained by scaling themotion vector MV using a time distance between pictures when theco-located block in a subsequent picture has a motion vector MV. Inother words, the forward motion vector MV_(f) and the backward motionvector MV_(b) are determined using the following Equations 1 and 2below:

$\begin{matrix}{{MVf} = \frac{{TRb}*{MV}}{TRd}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

$\begin{matrix}{{MVb} = \frac{\left( {{TRd} - {TRb}} \right)*{MV}}{TRd}} & {{Equation}\mspace{14mu} 2}\end{matrix}$where MV is the motion vector of the co-located block in the subsequentpicture, MV_(f) is the forward motion vector of the direct mode for a Bpicture, MV_(b) is the backward motion vector of the direct mode for theB picture, TRd is a time distance between the subsequent picture and areference picture pointed by the motion vector of the co-located blockin the subsequent picture, and TRb is a time distance between a Bpicture and a reference picture pointed by the motion vector of theco-located block in the subsequent picture.

In the direct mode, two motion-compensated blocks are obtained using thetwo motion vectors MV_(f) and MV_(b), and a prediction block is obtainedby averaging or interpolative calculation of the two motion-compensatedblocks.

SUMMARY OF THE INVENTION

The present invention relates to a method of determining motion vectorsfor a bi-predictive image block.

In one embodiment, reference index information from a co-located imageblock for the bi-predictive image block is obtained. The motion vectorsof the bi-predictive image block are determined based on a type ofreference picture referred to by the reference index information.

For example, in one embodiment, a first motion vector of thebi-predictive image block may be set equal to a motion vector of theco-located image block if the reference index information refers to along-term reference picture. Also, a second motion vector of thebi-predictive image block may be set equal to zero if the referenceindex information refers to a long-term reference picture.

In an embodiment, the first and second motion vectors of thebi-predictive image block may be determined by scaling the motion vectorof the co-located block if the reference index information does notrefer to a long-term reference picture.

In an embodiment, the motion vector of the co-located image block maypoint to the long-term reference picture if the reference indexinformation refers to a long-term reference picture.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 illustrates an example display order including B pictures;

FIG. 2 illustrates an example coding order associated with the displayorder of FIG. 1;

FIGS. 3A to 3B are flowcharts illustrating a method of coding a movingpicture sequence in a moving picture coding system according to anexample embodiment of the present invention;

FIG. 4 illustrates a method of coding a moving picture sequence in whicha scene change occurs according to an example embodiment of the presentinvention; and

FIG. 5 illustrates a method of coding a B picture in direct modeaccording to an example embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the example embodiments of thepresent invention that are illustrated in the accompanying drawings.Wherever possible, the same reference numbers will be used throughoutthe drawings to refer to the same or like parts.

To begin with, before an embodiment of the present invention isdescribed, in moving pictures having a scene change, a picture in whicha scene change occurs entirely in the picture is defined as a scene cutpicture and a picture in which a scene change occurs partially in thepicture is defined as a partial scene change picture.

FIGS. 3A and 3B are flowcharts illustrating a method of coding a movingpicture sequence in a moving picture coding system according to anexample embodiment of the present invention. Referring to FIGS. 3A and3B, pictures are sequentially input from a moving picture sequence(S111).

Kinds of pictures are determined (S114). In other words, it isdetermined whether the input picture is a P picture or a B picture.Here, in this embodiment of the present invention, it is assumed that acoding with respect to an intra picture is completed in advance.

If a picture is a P picture, it is determined whether or not a scenechange occurs in the P picture (S117). Here, the scene change isdetermined by comparing the P picture with a picture (P picture or Bpicture) displayed just before the P picture.

If the scene is changed entirely among the P pictures, the P picture isa scene cut picture. If the P picture is determined as the scene cutpicture, a coding is carried out with reference to a long-term referencepicture (S120).

If the P picture is not a scene cut picture, it is determined whether ornot the P picture is a partial scene change picture (S123).

If the P picture is a partial scene change picture, blocks contained inan area in which the scene is changed are coded with reference to thelong-term reference picture as in step S120 (S126).

Blocks contained in an area in which the scene is not changed are codedwith reference to a short-term reference picture (S129, S132).

Here, the long-term reference picture is a picture stored in a long-termreference buffer, and the short-term reference picture is a picturestored in a short-term reference buffer.

The short-term reference buffer is provided with a first-input,first-output (FIFO) buffer in which a picture first input is outputfirst, and the pictures coded a relatively short time ago are stored inthe short-term reference buffer.

The pictures coded a relatively long time ago are stored in thelong-term reference buffer. And, pictures of respective scene sets,e.g., an intra picture, the scene cut picture, the partial scene changepicture and the like are stored in the long-term reference buffer.

Next, an example of scene sets and scene changes will be described toassist in understanding principles of the present invention. It shouldbe understood that this is a non-limiting example. As shown in FIG. 4,an intra picture I0 that is first scene cut picture of a scene set A1, afirst scene cut picture P50 of a scene set B1 and a first partial scenechange picture P120 can be stored in the long-term reference buffer.Here, a scene set is a set of similar pictures. For example, suppose thepictures represent a discussion program where an announcer appears, thena panel A appears, then the announcer appears again and then the panel Aappears again. The scene where the announcer first appears is scene setA1, and the scene where the panel A subsequently appears is scene setB1. The scene where the announcer appears again is scene set A2, and thescene where the panel A appears again is scene set B2. As describedabove, when a scene change occurs, the P picture is coded in the intermode with reference to a short-term reference or a long-term referencepicture instead of being coded in the intra mode. This reduces theamount of the bits to enhance coding efficiency.

Description of the steps S117 to S132 in FIG. 3A will be made withreference FIG. 4. As shown in FIG. 4, if the P picture P200 to be codednow is the scene cut picture belonging to the scene set B2, theshort-term reference pictures stored in the short-term reference bufferare not used. The scene cut picture P200 is the first picture of thescene set B2, and the scene set of the scene cut picture P200 isdifferent from the short-term reference pictures such as P199, P198,P197, etc., belonging to the scene set A2. The similarity of the scenecut picture P200 and the short-term reference pictures belonging to thescene set A2 is not great (e.g., P200 is part of the scene of panel Aand P199 is part of the scene of the announcer in the above describedexample), and precise coding cannot be achieved from such referencepictures.

In this case, the P picture is coded in inter mode with reference to thereference pictures P50 and P120 belonging to a scene set B1, which is asimilar scene to the scene of scene set B2 (e.g., both are scenes ofpanel A in the above-described example).

On the other hand, if a partial scene change occurs in the P pictureP250, the coding is performed differently depending on two conditions.In other words, the blocks included in the area where a partial scenechange occurs are coded in inter mode with reference to the long-termreference pictures P50 and P120 stored in the long-term referencebuffer. The blocks included in the area where a partial scene changedoes not occur are coded in inter mode with reference to the short-termreference pictures P249, P248, P247, etc., stored in the short-termreference buffer.

As described above, after one P picture is coded, if a next pictureexists (S159), then the next picture is input (S111).

Returning to step S114, if the picture input in step S111 is a Bpicture, then five prediction modes (intra mode, forward mode, backwardmode, bi-predictive mode and direct mode) are tested and one of them isselected as an optimal coding mode (S135, S138). In this specification,the direct mode will be described mainly.

First, one block of the B picture is read (S141). Of course, the otherblocks can be read subsequently. Then, a kind or type of a referencebuffer storing a specified picture is examined (S142). Namely, the typeof reference picture (e.g., long or short) is examined.

The specified picture is determined of the earlier pictures than the Bpicture in the coding order regardless of the display order. In otherwords, the specified picture is one of the reference pictures used tocode the B picture. Therefore, the specified picture can be a short-termreference picture or a long-term reference picture. The referencepictures may be before or after the B picture in display order and theyare stored in the short-term reference buffer and/or stored in thelong-term reference buffer. If the specified picture is a long-termreference picture, the forward motion vector of direct mode for the Bpicture is set as a motion vector of the co-located block in thespecified picture. The backward motion vector of direct mode for the Bpicture is determined to be zero (S150). However, if the specifiedpicture is a short-term reference picture, the reference picture indexand the motion vector calculated at the co-located block in thespecified picture are read (S144). The reference picture index and themotion vector is calculated previously and stored in the system buffer.According to the reference picture index, it is determined whether themotion vector of the co-located block in the specified picture points toa long-term reference picture (S147). As described above, the short-termand long-term reference pictures are stored in the short-term referencebuffer and the long-term reference buffer, respectively.

If the motion vector of the co-located block in the specified picturepoints to the long-term reference picture, the B picture is coded usingthe following expressions 3 and 4 (S150):MVf=MV  Expression 3where MV is a motion vector of the co-located block in the specifiedpicture, and MVf is a forward motion vector of direct mode for the Bpicture; andMVb=0   Expression 4where MV is a motion vector of the co-located block in the specifiedpicture, and MVb is a backward motion vector of direct mode for the Bpicture

In other words, if the motion vector of the co-located block in thespecified picture points to the long-term reference picture, the forwardmotion vector in the direct mode for the B picture is the motion vectorof the co-located block in the specified picture and the backward motionvector is zero.

As shown in FIG. 5, in the step S150, if the motion vector of theco-located block in the specified picture P200 points to the long-termreference picture P50, TRd and TRb are meaningless in the conventionalexpressions 1 and 2.

Referring to FIG. 5, a more detailed description will be made. Wheninserting two B pictures B1 and B2 into a moving picture sequence andcoding them, the P picture P200 that is earlier than the B1 and B2pictures in coding order is coded first. Here, since the P picture P200is a scene cut picture in which a scene change occurs, the P pictureP200 is coded in inter mode from the long-term reference picture P50stored in the long-term reference buffer. According to the coding order,the next picture to be coded is the B1 picture. Since the B1 picturebelongs to the scene set A2, most blocks thereof are coded in a forwardmode from the short-term reference pictures belonging to the scene setA2 or in bi-predictive mode in which both of the two reference picturesbelong to the scene set A2. However, intra mode, backward mode orbi-predictive mode from the P picture P200 belonging to the other sceneset B2, and direct mode to obtain motion vectors of the direct mode fromthe co-located block in the P picture P200 are probably not used as thecoding mode for the blocks in the B1 picture.

Differently, since not only the B2 picture but also the specifiedpicture P200 used for motion vectors of the direct mode for the B2picture belong to the same scene set B2, the direct mode is selected asa coding mode for most blocks in the B2 picture. In other words, afterobtaining the motion vector of each block in the specified picture P200in the inter mode from the long-term reference picture P50 belonging tothe same scene set B2, the motion vectors of direct mode in the B2picture are calculated from the motion vector of the co-located block inthe specified picture P200. Since the B2 picture and the specifiedpicture P200 belong to the scene set B2, and the similarity between thescene set B1 to which the reference picture P50 belongs and the sceneset B2 is very high, the direct mode can be selected as a coding modefor most blocks in the B2 picture. Accordingly, the coding efficiencyfor the B2 picture is improved.

On the other hand, if the motion vector of the co-located block in thespecified picture points to a short-term reference picture, the Bpicture is coded using the conventional Expressions 1 and 2 (S153). Inthis time, since the short-term reference picture stored in theshort-term reference buffer belongs to the same scene set as the Bpicture and another scene set does not exist between the specifiedpicture and the short-term reference picture, the forward motion vectorand the backward motion vector of direct mode are determined using theconventional Expressions 1 and 2 related to TRd and TRb representingtime distance.

After one block of a B picture is coded, the next block (if it exists)in the B picture is read and coded subsequently (S156). Such processesare performed on the blocks in the B picture. After the B picture iscoded, the next picture (if it exists) is input (S159 and S111) andcoded so that a moving picture coding is achieved.

As described above, according to a moving picture coding method of thepresent invention, the forward motion vector and the backward motionvector in the direct mode for the B picture are determined differentlybased on the type of reference picture pointed to by the motion vectorof the co-located block in the specified picture. When coding the Bpicture, the direct mode is mainly used as the coding mode to enhancecoding efficiency.

According to the moving picture coding method of the present invention,the P picture in which a scene change occurs is coded in inter modeusing motion compensation from a long-term reference to reduce theamount of bits and enhance coding efficiency.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present invention. Thus,it is intended that the present invention covers the modifications andvariations of this invention.

What is claimed is:
 1. A method of determining and using motion vectorsfor a bi-predictive image block, the method performed by a movingpicture coding system and comprising: obtaining, by the moving picturecoding system, reference index information from a co-located image blockfor the bi-predictive image block; determining, by the moving picturecoding system, motion vectors of the bi-predictive image block based ona type of reference picture referred to by the reference indexinformation, the reference picture being for the co-located image block,the type being one of long term and short term, the type characterizinga temporal distance of the reference picture for the co-located imageblock with respect to the bi-predictive image block, the determiningstep determining the motion vectors according to a first set ofexpressions if the reference index information refers to a referencepicture of the short term type and the determining step determining themotion vectors according to a second set of expressions if the referenceindex information refers to a reference picture of the long term type,the second set of expressions being different than the first set ofexpressions; and decoding, by the moving picture coding system, thebi-predictive image block based on the determined motion vectors.
 2. Themethod of claim 1, wherein the second set of expressions sets a firstmotion vector of the bi-predictive image block equal to a motion vectorof the co-located image block.
 3. The method of claim 2, wherein thefirst motion vector of the bi-predictive image block is a forward motionvector.
 4. The method of claim 2, wherein the determining step isperformed in a direct mode of operation.
 5. The method of claim 2,wherein the first set of expressions determines the first and secondmotion vectors of the bi-predictive image block by scaling the motionvector of the co-located block.
 6. The method of claim 1, wherein thesecond set of expressions sets a motion vector of the bi-predictiveimage block equal to zero.
 7. The method of claim 6, wherein the motionvector of the bi-predictive image block is a backward motion vector. 8.The method of claim 6, wherein the determining step is performed in adirect mode of operation.
 9. The method of claim 6, wherein the firstset of expressions determines first and second motion vectors of thebi-predictive image block by scaling the motion vector of the co-locatedblock.
 10. The method of claim 1, wherein the second set of expressionssets a first motion vector of the bi-predictive image block equal to amotion vector of the co-located image block and sets a second motionvector of the bi-predictive image block equal to zero.
 11. The method ofclaim 10, wherein the first motion vector of the bi-predictive imageblock is a forward motion vector and the second motion vector of thebi-predictive image block is a backward motion vector.
 12. The method ofclaim 10, wherein the determining step is performed in a direct mode ofoperation.
 13. The method of claim 10, wherein the first set ofexpressions determines the first and second motion vectors of thebi-predictive image block by scaling the motion vector of the co-locatedblock.